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ABSTRACT 



The purpose of this review was to look for trends across 
time in response rates and variables studied for published mail surveys and 
to compare response rates and variables studied for different target 
populations. Studies were identified in databases in four fields: education, 
psychology, business and marketing, and sociology. A total of 225 articles 
published between 1931 and the present were selected and classified according 
to population group and a number of survey characteristics. Mean response 
rates show a cyclical pattern, with the 5-year period with the highest 
response rate being 1956 through 1960. There was a decline for the following 
10 years, with a rebound in 1971-75, followed by another drop and a rebound 
in 1986-90. Between 1991-95, the response rate was within one percent of the 
lowest mean percentage of any period studied. As journals lower their 
standards (response rate required) for publication, they might increase the 
number of articles accepted, and this may affect the mean response rate for 
the period. The overall response rate was affected by the disproportionate 
number of surveys of the general public. The study also suggests that more 
highly educated populations tend to have higher response rates . It was 
disappointing not to find a trend of increasing response rates as researchers 
learn more about the relative effectiveness of survey procedures. Why this 
should be so raises interesting research questions. (Contains 6 tables.) 
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Introduction 



Reviews of research studies on mail surveys have usually focused on identifying the survey 
methods most likely to produce high response rates. In order to obtain a body of studies that is large 
enough to examine different potential response facilitators, it has been necessary to utilize studies 
conducted over a period of years. While textbook authors (and doctoral committees) have standards of 
acceptable response rates, there has been no research to establish the response rates generally obtained in 
mail surveys and that are deemed acceptable for publication. The purpose of this review was to look for 
trends across time in response rates and variables studied for published mail surveys, and to compare 
response rates and variables studied for different target populations. 

Method 

The studies included in this review were identified using the terms "mail survey**"" and 
"response rate*" in searches of databases representing four fields: education, psychology, business and 
marketing, and sociology. Each study accepted for this review used a split sample approach with either a 
control or comparison group to isolate the effects of particular variables in the process of conducting a 
mail survey. A total of 225* articles published between 1931 and the present were selected. The 
following variables were recorded and are examined in this review: year of publication, response rate, 
variable being manipulated, and population. Some articles reported more than one independent study. 
Within a study, there sometimes were multiple variables and/ or populations (n). 

In this review, studies were classified by population group when the survey topic was relevant to 
their membership in that population. Those remaining were classified as "general public." All 
population groups except the general public are considered homogeneous in some respect, with the basis 
for grouping most frequently related to occupation. Some population groups had a sufficient number of 
studies with a specific occupation (college educators, for example) to form a group. In others, such as 
Nonprofessionals, it was necessary to combine studies involving similar population groups (mechanics, 
real estate salespeople, loan brokers, etc.). Each individual survey targeted a nonprofessional 
occupational group, but there were not enough studies using a specific occupation to formulate a group 
limited to that one occupation. 

The following manipulated variable categories were used: incentives, personalization, followups, 
appeals, precontact, postage (outgoing), return postage, length, format or appearance, organizational 
sponsor, signatory (person signing the cover letter), anonymity, paper color, deadline, topic salience or 
interest, order of questions or sections, commitment to participate, address (where questionnaire was 
sent), humor, questionnaire content, question type, endorsement, time cues, and other. 

*A basic list of articles can be found in ERIC Document # 402 318, updated via more recent searches of the 

relevant databases. 
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Resxilts 



The average response rate for each five-year period is shown in Table 1. The highest response 
rate was in the period from 1956-1960. There was a sudden increase in the number of studies of mail 
survey techniques in the early seventies, which has continued to the present. There was considerable 
variation in the range of response rates , with some of the lowest response rates of published articles in 
the most recent ten years. 

Table 1 

Mean Response Rate Across Time 



Time period 


Number of studies 


Mean response rate 


Response rate range 


1931 - 1950 


5 


47.08% 


18.0% - 73.5% 


1951 - 1955 


8 


43.86% 


21.9% -73.1% 


1956 - 1960 


15 


62.13% 


17.2% -99.0% 


1961 - 1965 


9 


52.86% 


34.0% - 95.0% 


1966 - 1970 


16 


40.67% 


18.2% -86.3% 


1971 -1975 


40 


54.00% 


17.5% -88.8% 


1976 - 1980 


47 


47.67% 


17.4% -82.2% 


1981 - 1985 


37 


42.00% 


16.5% -79.0% 


1986 - 1990 


49 


48.58% 


03.5% - 91.0% 


1991 - 1995 


36 


41.83% 


03.6% - 75.2% 



Twelve publication sources were found to have published methodological articles on an ongoing 
basis (more than five studies apiece). Table 2 shows that mean response rates for these sources varied 
from 27% to 64%. Response rates of individual survey articles that had been published ranged from a 
low of less than 4% to a high of 95%. 

The publication data for articles published by these journals in the 1990s are presented in Table 3. 
(Four of the journals in Table 2 had not published any such articles in the 1990s.) Public Opinion 
Quarterly, Journal of Marketing, Research in Higher Education, and the Journal of Experimental 
Education appear to have become more selective. ERIC and the Journal of the Market Research Society 
have remained fairly consistent, while Psychological Reports had a considerably lower mean response 
rate (16% compared with 41% overall in Table 2) and upper limit of the range (49% compared with 81%). 
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Table 2 



Publication of Methodological Articles by Journals on an Ongoing Basis 



Journal Publication range N Mean response rate Response rate range 



Public Opinion Quarterly 


1954 - 1993 


33 


63.6% 


26.5% - 95.0% 


Journal of Experimental Education 


1983 - 1991 


8 


59.0% 


16.5% -80.1% 


ERIC 


1973 - 1994 


15 


57.8% 


25.8% - 80.0% 


Research in Higher Education 


1974 - 1990 


7 


57.6% 


36.7% - 82.2% 


Journal of Applied Psychology 


1940 - 1977 


17 


55.0% 


18.0% - 88.8% 


Journal of Advertising Research 


1962 - 1984 


16 


42.2% 


26.5% - 69.0% 


Journal of the Academy of 










Marketing Science 


1978 - 1989 


7 


42.0% 


26.5% - 69.0% 


Journal of Marketing 


1952 - 1995 


12 


41.9% 


26.0% - 73.3% 


Psychological Reports 


1971 - 1994 


11 


40.7% 


3.6% - 80.5% 


Journal of Marketing Research 


1966 - 1982 


28 


38.8% 


17.8% - 70.0% 


Journal of the Market Research 










Society 


1972 - 1995 


19 


32.8% 


9.0% -65.7% 


Industrial Marketing Management 


1990 - 1995 


11 


27.3% 


4.4% -51.3% 



Table 3 

Publications from 1990 - 1995 



Journal 


Publication range 


N 


Mean response rate 


Response rate range 


Public Opinion Quarterly 


1990 - 1993 


3 


75.4% 


67.0% -91.0% 


Journal of Marketing 


1995 


1 


64.8% 




Research in Higher Education 


1990 


1 


64.0% 




Journal of Experimental Education 


1991 


2 


60.0% 


48.3% - 71.7% 


ERIC 


1990 - 1994 


4 


57.2% 


25.8% - 77.3% 


Journal of the Market Research 










Society 


1990-1995 


7 


31.2% 


14.2% -65.7% 


Industrial Marketing Management 


1990-1995 


11 


27.3% 


4.4% -51.3% 


Psychological Reports 


1991 - 1994 


2 


26.3% 


3.6% - 49.0% 



ERIC 
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Mean response rates for various population groups are documented in Table 4. Medical 
professionals (doctors, dentists, and psychologists) had the highest response rate, 8% higher than the 
closest other group. College alumni, other medical professionals, and K-12 educators responded at 
similar levels (from 62% to 64%). Other professionals bridged the 8% drop to the next groups four groups 
that clustered from 51% to 54%. Below the 50% mark and approximately 8% down from the next highest 
group were customers, closely followed by business executives and the general public (from 40% to 43%). 
Nonprofessional occupational groups, farmers and ranchers, and engineers had the lowest response rates 
(31% to 34%). 

Table 4 

Average Response Rate by Population 



Population N Mean response rate Range 



Medical doctors (doctors, dentists, psychologists) 
College alumni 

Other medical (nurses, physical therapists, social 
workers, rehab counselors, mixed 
K-12 educators 
Other professionals 
Special interest groups 
Postsecondary educators 
College students 
Employees 
Customers 

Businessmen (executives, owners, managers, 
administrators) 

General public 

Nonprofessionals, targeted by occupation 

Farmers and ranchers 

Engineers 



n 


71.8% 


59.3% - 97.0% 


21 


63.7% 


25.8% - 90.0% 


6 


62.8% 


36.2% - 80.5% 


11 


62.1% 


25.2% - 80.0% 


7 


58.0% 


18.8% -80.1% 


22 


54.3% 


14.2% - 94.7% 


12 


54.1% 


27.0% - 86.5% 


12 


52.0% 


23.2% - 86.3% 


13 


51.3% 


15.4% - 81.4% 


17 


42.5% 


10.5% - 73.3% 


23 


41.7% 


14.6% - 99.0% 


76 


39.7% 


7.5% - 75.2% 


16 


33.7% 


3.5% - 68.3% 


4 


31.4% 


17.5% - 66.9% 


7 


30.8% 


4.4% - 66.7% 



Note. Does not include the following survey populations: students (1), high school students (2), college 
dropouts (2), unknown (2) 



Table 5 shows the frequencies with which specific variables occurred across time. Twelve 
variables or types of variables have been the subject of more than eight studies and appear in the upper 
part of the table. Incentives have been investigated most frequently overall and more often in the years 
since 1980 than before. Research studies on incentives, appeals, postage, and followups were published 
in each of the time periods, and on personalization and precontacts in all but one time period each. 

Return postage, length, format, format, organizational sponsor, anonymity, and cover letter signatory 
have shown consistent research efforts since 1971-75, with some of them having been investigated 
sporadically prior to that. There is relatively little evidence regarding the effects of the remaining 
variables. 

Studies on personalization (and follow-ups to a lesser extent) peaked in the early seventies, 
appeals in the late seventies, and incentives in the late nineties. Precontacts were studied at the 
approximately the same level throughout the seventies and eighties. The attention to questionnaire 
length was most noticeable in the late 1980s and to questionnaire format in the late 1980s and early 90s. 
Almost all of the research on deadlines was conducted in the late 1970s, as well as the largest number of 
studies of organizational sponsors. 

Some population groups have been researched more thoroughly than others regarding the effect 
of various mail survey manipulations. All of the 12 major variables have been investigated with the 
general public and only one population group, college alumni, as shown in Table 6. The largest number 
of studies of the general public have focused on incentives. Over one third of the studies on incentives, 
appeals, and precontacts have been done on this group. Investigations with alumni were most frequently 
regarding personalization and format. 

Special interest groups and customers have each been the study for all but one of the major 
variables, college students all but two. Several of the population groups cited in Table 6 have been the 
subject of only sporadic investigations for a limited number of variables. 

Discussion 

One might expect that as the amount of research increased through the years those conducting 
mail surveys gained more information about techniques that improve response rates, the mean response 
rate would increase linearly. That has not been the case. Mean response rates show a cyclical pattern. 

The five-year period with the highest mean response rate was from 1956 - 1960. There was a decline 
during the following 10 years before rebounding in the 1971-75, when the number of such studies being 
reported more than doubled, only to descend again the following decade. There was another rise in 1986- 
1990 before dropping again in 1991-95 to within one percent of the lowest mean percentage of any period 
from 1951 to the present. Each time the average response rate rose to a new peak, the rate was lower than 
during the previous peak period. 

As journals lower their standards (response rate required) for publication, they might well 
increase the number of articles accepted. They then accept a larger number more articles (with lower 
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Table 5 



Variables Studied by Time 





pre 

1950 


1951- 

1955 


1956- 

1960 


1961- 

1965 


1966- 

1970 


1971- 

1975 


1976- 

1980 


1981- 

1985 


1986- 

1990 


1991- 

1995 


Total 


Incentives 


2 


2 


5 


4 


1 


5 


11 


13 


27 


13 


83 


Personalization 


2 


2 




3 


6 


13 


8 


6 


7 


5 


52 


Appeals 


1 


2 


3 


1 


1 


2 


12 


4 


4 


5 


35 


Precontacts 




2 


2 


1 


4 


6 


5 


5 


7 


3 


35 


Postage 


1 


2 


5 


2 


1 


5 


4 


4 


2 


3 


29 


Followups 


1 


1 


2 


1 


2 


6 


3 


3 


5 


1 


25 


ReturnPostage 






2 


3 




6 


4 


5 


2 


1 


23 


Length 


1 




1 


4 




3 


3 


2 


6 


2 


22 


Format 




1 






2 


2 


3 


1 


5 


6 


20 


Organizational 

Sponsor 






2 


1 




2 


6 


1 


2 


5 


19 


Anonvmitv 




2 




1 


1 


2 


4 


4 


3 


1 


18 


Sienatorv 












3 


4 


1 


4 


2 


14 


Color 








1 




2 


1 


1 




2 


7 


Deadline 














6 


1 






7 


Topic, Interest 
Salience 


1 










1 


1 


1 




4 


7 


Order 

(hard/easv,etc.) 












1 








2 


3 


Commitment 












1 


1 




1 




3 


Address 
(where sent) 










1 




1 


1 






3 


Humor 












1 


1 


1 






3 


Ore Content 








1 




1 


1 








3 


Ouestion tvpe 












2 




1 






3 


Endorsement 














1 






1 


2 


Time Cue 
















2 






2 


Mi sc 














3 




1 


1 


5 
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Table 6 



Variables Studied by Population 





MD 


Alumni 


Other 

Medical 


K-12 

Eductrs. 


Other 

Prof. 


Special 

Int. 


Postsec. 

Eductrs. 


College 

Students 


Employee* 


Incentives 


2 


3 


1 


1 


3 


3 


3 


4 


1 


Personalization 


3 


6 


2 


3 


1 


3 


6 


2 


2 


Appeals 


2 


1 


2 


2 




3 


2 


1 


1 


Precontacts 




3 


1 






6 




2 




Postage 


4 


3 


1 


1 


1 


5 






1 


Followups 


1 


3 


2 






1 




3 


1 


Return Postage 


1 


1 


1 


1 


1 


1 




1 




Length 


1 


1 




3 




1 


2 


2 


1 


Format 


1 


5 




2 




1 


1 


1 


2 


Organizational 

Sponsor 




1 








3 






1 


Anonymity 


1 


1 




2 


2 


1 




1 


3 


Signatory 




1 






1 




3 


1 




Color 




2 










1 




1 


Deadline 


1 








1 






1 




Topic, Interest 
Salience 




1 








2 




1 




Order 

(hard/easv.etc.) 












1 








Commitment 


















2 


Address 
(where sent) 




















Humor 














2 






Ore Content 












1 








Ouestion type 


















1 


Endorsement 




1 






1 










Time Cue 














1 






Mi sc 








1 




1 


1 
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Table '6 (continued) 





Customer* 


Business 


Gen. 

public 


Non- 

prof. 


Farmers 

Ranchers 


Engineers 


H.S. 

Students 


Misc. 


Incentives 


6 


9 


29 


8 


2 


6 


1 


1 


Personalization 


6 


2 


12 


1 






1 


2 


Appeals 


1 


6 


13 






1 






Precontacts 


3 


1 


15 


3 




1 






Postage 


1 


2 


7 


2 






1 




Followups 


3 




8 


1 


1 




1 




ReturnPostage 


2 


2 


9 


1 


1 






1 


Length 


1 




8 






1 


1 




Format 


1 


1 


4 


1 










Organizational 

Sponsor 




3 


8 


1 


1 


1 






Anonymity 


2 




4 










1 


Signatory 


1 


2 


1 


2 




1 


1 




Color 




2 






1 








Deadline 




3 


1 












Salience 






3 












Order 

(hard /easy ,etc.) 


1 












1 




Commitment 




1 














Address 
(where sent) 




2 








1 






Humor 




1 














Ore Content 


1 




1 












Ouestion type 




1 




1 










Endorsement 


















Time Cue 






1 












Misc 






2 
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rates), affecting the overall mean response rate for the time period. For example. Industrial Marketing 
Management published the largest number of articles in the 1990-1995 period, almost one third of the 
total number for that time period, but had a low mean response rate of 27.3% and accepted articles with 
response rates as low as 4.4%. 

Population groups were not equally represented in this study. The overall response rate is 
affected by the disproportionate number of surveys of the general public, which had one of the lower 
overall means, providing yet another possible interactive effect on response rate. Another researcher 
might use other population groups or subsume some of the smaller groups into others. The present 
study, however, attempted to preserve the individual nature of the studies as much as possible. 

It has been suggested that individuals with more education respond at higher levels than those 
with less education, and that homogeneous groups respond at higher levels than heterogeneous ones. 
This study tends to support both suppositions. More highly educated populations did, in general, have 
higher response rates. But this was not uniform; two exceptions were noted. Engineers, classified as 
professionals, had the lowest response rate of any group; and special interest groups (homogeneous 
populations targeted on a basis other than occupation, such as bowlers, farm show attendees, etc.) when 
surveyed on a topic relevant to that special interest, had higher response rates than college educators and 
college students. Education was neither uniform nor identifiable in groups such as special interest 
groups, employees, customers, businessmen, farmers and ranchers, and the general public. 

Considering the numerous studies that have been done with surveys targeting the general public, 
several variables have not been investigated (or not extensively so), while interest in incentives continues 
to the point that one might wonder 'Tiow much is enough?" Despite being the most heavily studied 
population, surveys of the general public were among those with low overall response rates. 

It was disappointing to fail to find a consistent pattern for response rates through the years, 
particularly a pattern of increasing response rates. The population has changed over the years, with more 
people being employed outside the home, having less leisure time at home to respond to mail surveys. 
Mail (as well as telephone) surveys abound today and have become commonplace, so that they may 
trigger a "familiarity breeds contempt" reaction in the recipient. This study does not profess to answer 
the question of why response rates have not continued to increase as researchers learned more about the 
relative effectiveness of various procedures.. 

As with most (if not all) studies, there are limitations with this one. The studies that were used 
were neither a random sample nor an exhaustive group of all possible studies. The studies were those 
that had been conducted to investigate mail survey procedures using a split-sample approach. 
Presumably such studies were undertaken by individuals more knowledgeable about survey methods 
than the general population of researchers or individuals who conduct surveys. In effect, other than the 
variables that were being manipulated, the methods used in their surveys would have would been 
expected to have produced the highest response rates possible. 




9 



Also, there are many surveys that are conducted in which there is no manipulated variable. 
Another limiting factor is that only published studies were used. As was documented in this study, some 
journals have fairly high standards for acceptance. Early studies are difficult to locate because they are 
not in computer searchable databases. In essence, this study has addressed some questions but 
formulated new ones in the process. 
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